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he dissemination of 
valuable and novel 
scientific information 
provides the pulse for bio- 
medical publishing. Scientific 
journals catalog the contribu- 
tions, thoughts, and opinions 
of researchers, investigators, 
and experts in the field. Au- 
thors consider the reputation 
and quality of a journal prior 
to submitting a manuscript for 
consideration. It is reasonable 
to think that readers also con- 
sider journal prestige as a fac- 
tor in journal selection. The 
prestige of a journal depends 
on the validity, usefulness, 
and quality of the articles pub- 
lished. This article will define 
and examine the peer-review 
process as well as explore the 
roles and responsibilities of 
the peer reviewer. 

The Peer-Review Process 

Aside from its use in sci- 
entific journals, peer review 
is the process by which grants 
are allocated, academics are 
promoted, textbooks are writ- 
ten, and Nobel prizes are won 
(Smith, 2006). A publication 
that has been peer reviewed 
gains respectability and accep- 
tance and is considered a rel- 
evant contribution to the field. 
Publication in a peer-reviewed 
journal is an important crite- 



rion for admissibility of scien- 
tific evidence in courts of law 
(Kumar, 2009). The basis of the 
peer-review process is the ac- 
ceptance of written investiga- 
tional findings from an author 
or group of authors that are 
then forwarded to a group of 
experts (referees) in the field 
for assessment of their quality, 
accuracy, relevance, and nov- 
elty (Shuttleworth, 2009). Tra- 
ditionally, these experts are not 
paid for their opinions and are 
not part of an editorial staff. 

The goal of peer review is 
to determine if an article should 
or should not be published and 
to improve the article before 
publication (Neale & Bowman, 
2006). It is a process that entails 
filtering out manuscripts that 
are misleading, irrelevant, inac- 
curate, or that contain poten- 
tially harmful content (Kumar, 
2009). Once the peer-review 
process is complete (see Figure 
1), the editor of a journal bears 
responsibility for its content 
and may choose to agree or dis- 
agree with the opinions of the 
reviewers (Garmel, 2010). 

Limitations 

Despite its acceptance as a 
critical part of quality control, 
peer review is not a perfect pro- 
cess. In 2003, The Cochrane Col- 
laboration published a review 



concluding that there is little evi- 
dence to support the use of edito- 
rial peer review as a mechanism 
to ensure quality of biomedical 
research, despite its widespread 
use and costs (Jefferson, Ru- 
din, Brodney Folse, & Davidoff, 
2007). There are few published, 
randomized controlled studies 
relating to peer review; therefore 
it remains ill- defined. 

The peer-review process 
can be time consuming, costly, 
subject to reviewer bias, and 
inept at identifying fraudulent 
manuscripts. A well-known ex- 
ample of the failure of peer re- 
view is the publication of two 
fraudulent papers by Hwang 
Woo-Suk concerning stem cell 
research in the journal Science 
(Kumar, 2009). 

In addition, there are no 
agreed-upon evidence-based 
guidelines as to what constitutes 
a qualified reviewer. A study 
examining the relationship of 
previous training and experi- 
ence of journal peer reviewers 
to subsequent review quality 
determined that no identifiable 
types of formal training or ex- 
perience predicted reviewer 
performance. The authors sug- 
gest that journals implement 
routine review rating systems to 
periodically monitor the quality 
of their reviews (Callaham & 
Tercier, 2007). 
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Editor receives manuscripts 
(manuscripts may be numerous) 



Editor or associate editor may filter 
out unacceptable manuscripts 



Manuscripts sent to peer review 
team members for consideration of 
acceptability and feedback with a goal 
for improvement 
V 



Peer reviewers accept, accept with 
revisions, or reject manuscripts 



Manuscripts returned to the editor 
with suggestions for improvement 



Final acceptance and decision to 
publish made by the editor 
(final authority) 



c 



Publication 
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Figure 1. Key steps in the 
peer-review process. 

Traditionally, the peer- 
review process has been con- 
ducted anonymously, with au- 
thor and reviewer identities 
masked during the review pro- 
cess. Although this may pro- 
tect reviewers from author de- 
mands and retaliation, reviewer 
anonymity is being debated and 
is under increasing scrutiny 
(Garmel, 2010; Leek, Taub, & 
Pineda, 2011). Early evidence 




Use your smartphone to access 
the CONSORT Statement and the 
EQUATOR Network's resources 
for editors and peer reviewers. 
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supporting blind peer review 
(McNutt, Evans, Fletcher, & 
Fletcher, 1990) was later chal- 
lenged by studies suggesting 
that such a practice made no 
editorially significant differ- 
ence to review quality, publica- 
tion recommendation, or time 
taken to review, but did increase 
the probability of reviewers de- 
clining to review (van Rooyen, 
Godlee, Evans, Smith, & Black, 
1998; Justice, Cho, Winker, Ber- 
lin, & Rennie, 1998; van Rooyen, 
Godlee, Evans, Black & Smith, 
1999). It is possible that an open 
process may increase coopera- 
tion between reviewers and au- 
thors and lead to a decreased 
risk of reviewing errors (Leek, 
Taub, & Pineda, 2011). 

Some journals have already 
considered transition to open 
peer review. In 1999, the Brit- 
ish Medical Journal adopted an 
open (signed) review system that 
remains in place today. Most re- 
cently, the journal has examined 
the effect of notifying reviewers 
that their signed reviews might 
be posted on the web. Their con- 
clusion was that alerting peer re- 
viewers that their signed reviews 
might be available in the public 
domain on the journal's website 
had no important effect on review 
quality but was associated with 
a high refusal rate (van Rooyen, 
Delamothe, & Evans, 2010). Other 
journals such as Nature and The 
Public Library of Science are revis- 
ing old review criteria, creating 
open access, and 
examining public 
review (Editors of 
The New Atlantis, 
2011). 

One study ex- 
amined the effects 
of adding a statis- 
tical peer reviewer 



and using a checklist of manu- 
script quality. The study showed 
a positive effect when a statistical 
reviewer was added to the field- 
expert peers, but no statistically 
significant positive effect was 
suggested by the use of report- 
ing guidelines (Cobo et al., 2007). 
Additional alternative methods 
of peer review such as open peer 
review without suppression of 
publication, postpublication re- 
view, a hybrid system (traditional 
with postpublication review), 
author-suggested peer review, 
author model of peer review, and 
peer review consortia have been 
discussed and explored in the lit- 
erature (Kumar, 2009). 

Reviewer Responsibilities 

However ill- defined it may 
be, the peer-review process is 
still the gold standard that will 
continue to drive scholarly pub- 
lication. Understandably, a large 
part of the responsibility for the 
success or failure of the peer- 
review process depends upon 
peer reviewers. A peer reviewer 
should be both a scholar and a 
scientist with complex analyti- 
cal skills, which allows for the 
critical analysis of data in the 
interest of improved outcomes 
(Bearinger, 2006). 

Peer review can be time 
consuming and laborious; 
therefore, accepting the respon- 
sibility of peer review requires 
commitment on the part of the 
reviewer. It should be viewed 
as a professional responsibil- 
ity, not to be taken lightly, given 
that the end result determines 
what is relevant, in print, to a 
specific body of knowledge. Just 
as editors and journals respect 
their reviewers, often acknowl- 
edging their contributions pub- 
lically reviewers should respect 
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the editor and the journal by 
producing a quality of work that 
is consistent with the journal's 
reputation and integrity. 

Just as a surgeon would 
prepare for surgery, a reviewer 
must prepare for a review. First, 
it is important to understand a 
selected journal's mission and 
review criteria as they will be 
incorporated into manuscript 
review. Once an invitation to 
review is accepted, reviewers 
normally agree to complete the 
assigned manuscript review 
within a specified time frame. 
This is not only important to 
journals and editors who have 
publication deadlines, but to 
authors who eagerly await news 
of acceptance or rejection. Time 
is especially important in cases 
where the author is asked to 
consider recommended revi- 
sions prior to a final decision 
of acceptance or rejection. Sec- 
ond, reviewers must maintain 
confidentiality; using any infor- 
mation gained for self-interest 
or extracurricular professional 
discussion is unethical. 

Given that a reviewer's au- 
thority to recommend a manu- 
script's acceptance or rejection 
carries weight with an editor's 
final publication decision, care- 
ful consideration of the manu- 
script and each individual sec- 
tion is required prior to any 
such recommendation. A fair 
analysis requires a reviewer 
to have undisturbed focus, a 
discerning eye for detail, and 
knowledge of appropriate sec- 
tional content (see Table 1). It 
is important to consider if the 
information is accurate, un- 
derstandable, valid, useful, 
and transparent. Grammar is 
important, and errors can be 
pointed out; however, the main 



Table 1. Sectional Content for Manuscript Review 



Abstract 

Brief, comprehensive summary of article contents 
Written in clear, concise language 

Includes the most important concepts, findings, implications 
Usually the first article contact for readers 

ntroduction 

Presents problem or concept under study 
Describes research strategy (not design) 
States hypotheses 

Method 

Meticulous description of how the study was conducted 
Includes study criteria, variables, operational definitions 
Detailed enough to provide for study replication 

Results 

Summarizes data and data analysis 
Includes results that are counter to positive study 
Good place for tables, graphs, charts for clarity 

Discussion 

Opens with statement of support or nonsupport for hypothesis 
Explains and qualifies results 
Allows for inferences and conclusions 
States theoretical or practical consequences of results 

Conclusion 

Summary of the problem, findings, implications 
Brief, concise, direct 
Conclusion supported by article data 

References 

Conform to journal expectations/format 
Acknowledges previous scholarly work 
Information provides easy location of sourced material 

Appendices 

Appropriate for brief material easily presented in print format 
May include headings or subheadings 

Tables and figures 

Supplement not duplicate text 
Not appropriate for small amounts of data 
Class of information should be mentioned in the text 



Note. Adapted from American Psychological Association (2010). 



concern for the reviewer is rel- 
evancy of manuscript content. 

Table 2 provides a list of 
important questions to con- 
sider when reviewing a manu- 
script. A helpful resource to 
guide review is the CONSORT 
Statement. Updated in 2010, it 
provides guidance for report- 
ing all randomized controlled 
trials (CONSORT, 2010). An 
additional resource is the 
EQUATOR Network (2012), 



an international initiative that 
seeks to improve the reliability 
and value of medical research 
literature by promoting trans- 
parent and accurate reporting 
of research studies. 

All reviewers are subject to 
bias. Gender, patriotism, and 
linguistic preference have been 
shown to affect peer review 
(Kumar, 2009). Reviewers are 
more likely to favor manuscripts 
that are clearly written, are cre- 
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Table 2. Important Questions to Consider When Reviewing a Manuscript 

Does the manuscript present novel or important information? 

Is the information relevant to the body of knowledge? 

Is the information presented accurate and evidence-based? 

Are references provided and what is the quality of the references? 

Is the writing clear, concise, and logical? 

Are manuscript structure and content formatted properly, including tables/figures? 

Is the abstract descriptive of the message in the paper? 

Are any bias or ethical concerns identified? 

Are there any areas that could benefit from further explanation? 

Are there any areas that could be deleted? 

If research based, does the information presented allow for experiment duplication? 



Table 3. Reviewers' Responsibilities to Authors 



Provide written, honest, and unbiased feedback in a timely manner 

Express a critical opinion about the manuscript, as experts in the field, in a collegial and constructive manner 

Comment on the style of writing, especially its clarity 

Rate the work's detail, methodology, relevance, accuracy, and originality 

Avoid comments or criticisms of a personal nature 

Maintain professionalism and confidentiality, especially given the competitive nature of research, funding availability, 
and publication 

Refrain from directly contacting authors without permission from the editor, unless the journal stipulates otherwise 



Note. Repinted, with permission, from Garmel (2010). 



ative, demonstrate positive re- 
sults, and have interesting titles, 
meanwhile rejecting manu- 
scripts with negative results, 
multiple errors, and seasoned 
information (Garmel, 2010). It 
is possible that senior reviewers 
may reject their juniors; manu- 
scripts from more prestigious 
institutions maybe more readily 
accepted than those from less- 
er-known institutions (Kumar, 
2009). Reviewers are respon- 
sible for disclosing biases that 
may hinder an impartial and 
balanced review. Lack of exper- 
tise in an area may not hinder 



review as useful comments may 
still be collected, but in this cir- 
cumstance, the editor should be 
informed that a lack of exper- 
tise exists (Garmel, 2010). 

Once the review is complete, 
reviewers offer scholarly input 
with the intent to improve the 
manuscript. Feedback should 
be constructive and the critique 
professional and positive. When 
a reviewer provides feedback 
that enables authors to revise 
and resubmit a publishable pa- 
per, the peer-review process is 
working as intended (Bearing- 
er, 2006). Length of the review 



is not as important as detailed 
suggestions for improvement. 
The review should begin with a 
recommendation for rejection, 
acceptance with minor revi- 
sions, or acceptance with major 
revisions. The reviewer should 
comment on the manuscript as 
a whole, then provide input on 
each individual section. Sugges- 
tions should be clear and provide 
direction. Comments should be 
detailed enough to assist authors 
with revisions but not so detailed 
that the manuscript is rewrit- 
ten (Garmel, 2010). Reviewers 
should remember to comment 
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on the appropriateness of the 
abstract and be certain it mirrors 
the content of the manuscript. 

Reviewing provides an op- 
portunity for learning and gain- 
ing exposure to cutting- edge 
research (Bearinger, 2006). Re- 
viewing is a skill that requires 
critical thinking; it will improve 
with time, practice, personal 
research, and writing. A good 
reviewer is competent, knowl- 
edgeable, unbiased, objective, 
punctual, consistent, ethically 
sound, constructive, and main- 
tains confidentiality (Garmel, 
2010; Kumar, 2009). 

Feedback 

Reviewers, like authors, 
can benefit from feedback; they 
should welcome input from 
editors and experienced col- 
leagues. Feedback is important 
for both new and seasoned re- 
viewers. Editors at a specialty 
journal in the top 11% of the 
Institute of Scientific Informa- 
tion's bibliographic database 
(ranked by number of citations) 
performed a 14-year longitudi- 
nal study designed to evaluate 
change in the review quality of 
individual peer reviewers. The 
study found that over time most 
journal peer reviewers received 
lower quality scores for article 
assessment. Proposed reasons 
were cognitive changes, com- 
peting priorities, or escalating 
expectations (Callaham & Mc- 
Culloch, 2011). Although it is 
not common practice, results 
such as these suggest that on- 
going self- evaluation by the re- 
viewer and validated reviewer 
evaluation on the part of the 
editor are important factors for 
ensuring quality peer review. 

Reviewing is a professional 
privilege, and reviewers are 



Table 4. Reviewers' Responsibilities to Editors 



Respond to the editors promptly if unable or unavailable to review a 
manuscript 

Recommend names of other experts as potential reviewers if unavailable 

Determine the scientific merit of the submission, with recommendations for 
acceptance or rejection 

Identify opportunities to improve the manuscript 

Point out potential ethical concerns about research methodologies or 
similarities with other papers or ongoing research 

Acknowledge personal or author conflicts of interest and inform the editor 
of these 



Note. Adapted, with permission, from Garmel (2010). 



Table 5. Reviewers' Responsibilities to Readers 



Ensure that published articles adhere to journal standards, as well as to 
standards of scientific practice 

Protect readers from incorrect or flawed research 

Identify missed references or erroneous citations (including misquoting or 
misinterpreting an author's findings) 



Note. Adapted, with permission, from Garmel (2010). 



advised to remember they are 
representing a journal and have 
responsibilities to authors (see 
Table 3), editors (see Table 
4), and readers (see Table 5). 
Perhaps most importantly, re- 
viewers are accountable to the 
medical community and the 
scientific body of knowledge 
impacted by their reviews. 

Conclusion 

While it is not a perfect pro- 
cess, traditional peer review 
remains the gold standard for 
evaluating and selecting qual- 
ity scientific publications. Ad- 
ditional research and the de- 
velopment of evidenced-based 
guidelines are needed to gov- 
ern this process, which is ex- 
pected to evolve in the future. 
Peer review is both an art and 
a science largely dependent on 



the quality of its review body. 
Competent peer reviewers are 
experts in their field account- 
able to authors, editors, read- 
ers, and the medical community. 
Peer reviewers act as advocates, 
or referees, for authors and en- 
able editors to make quality 
publication decisions. Peer re- 
view is a professional privilege 
and responsibility that directly 
impacts what is accepted as im- 
portant to a body of knowledge. 
Although the peer-review pro- 
cess can be time consuming and 
underappreciated, rewards such 
as mentorship, learning, expo- 
sure to cutting- edge research, 
and personal development make 
it a worthwhile investment. 
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